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(57) ABSTRACT 

An image processing circuit includes a processor that 
receives at least a portion of an image including a boundary 
and first and second contiguous pixels disposed on opposite 
sides of the boundary, the first and second pixels having 
respective first and second pixel values. The processor 
generates a boundary value from the first and second pixel 
values, compares the boundary value to a comparison value, 
and reduces the difference between the first and second 
values if the boundary value has a specified relationship to 
the comparison value. For example, such a processing 
circuit can be used to reduce blodciness in an image that has 
undergone block-based digital compression. 
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IMAGE PROCESSEVG CIRCUIT AND ' ' pixels 12. In the MPEG standards, a macro block is always 

METHOD FOR REDUCING A DIFFERENCE 16x16 pixels, although other compression sUndards may 

BETWEEN PIXEL VALUES ACROSS AN ^ °oacro blocks having other dimensioos. In the original 

IMAGE BOUNDARY video frame, i.e., the frame before conipression, each pixel 

5 12 has a respective liiminance value Y and a respective pair 

This is a continuation of prior appUcation Ser. No. °^ ' difference values and C^. 

09/20U70, filed Nov. 30, 1998, now U.S. Pat. No. 6,236, , Refeinng to FIGS. lA-lD, before comp^on of the 

754 issued May 22. 2001, the benefit of the filing date of digital luminance (Y) and chroma-difference (C^ 

which is hereby claimed under 35 USC 120. ""'^ ^"1"'* ^^'''T'""'- ' .'v 

JO pre-compression values, are generated from the onginal Y, 

TECHNICAL FIELD C^, and C^ values of the original frame. In the MPEG 4:2:0 

format, ib& pre-compression Y values are the same as the 

The invention relates generally to electronic and computer onginal Y values. Thus, each pixel 12 merely retains its 

circuits, and more particularly to an image processing circuit ^^^^^ luminance value Y But to reduce the amount of data 

and a method for reduang the difference between the to be compressed, the MPEG 4:2:0 format aUows only one 

respective values of a first pixel on one side of an image pre^rompression C^ value and one pre^ompression C^ 

boundary and a second pixel on the o^er side of the ^^^^ ^^^^ ^4 ^^^^ ^ ^ach of these 

boundary. For example, such a circuit and method can be prencompression C^ and C^ values are respectively derived 

used to reduce blockiness m an image that has undergone ^^^^^^ ^^^^^ fo^ ^^^^ ^ ^ 

block-based digital compression. respective group 14. Thus, referring to HGS. IB-ID, the 

BACKGROUND OF THE INVENTION pre-compression Y, C^, and Cr values generated for the 

macro block 10 are arranged as one 16x16 matrix 16 of 

To electronically transmit a relatively high-resolution pre-compression Y values (equal to the original Y value for 

image over a relatively low-band-widtb channel, or to elec- each pixel 12), one 8x8 matrix 18 of pre-compression C^ 

tronically store such an image in a relatively small memory ^ values (equal to one derived C^ value for each group 14 of 

space, it is often necessary to compress the digital data that four pixels 12), and one 8x8 matrix 20 of pre-compression 

represents the image. For example, High-Definition- C^ values (equal to one derived C^ value for each group 14 

Television (HDTV) video images are compressed to allow of four pixels 12). It is, however, common in the industry to 

their transmission over existing television channels. Without call the matrices 16, 18, and 20 "blocks" of values, 

compression, HDTV video images would require transmis- ^ Furthermore, because it is convenient to perform the com- 

sion channels having bandwidths much greater than the pression transforms on 8x8 blocks of pixel values instead of 

bandwidths of existing television channels. Furthermore, to 16x16 blocks, the block 16 of pre-compression Y values is 

reduce data trafSc and transmission time to acceptable subdivided into four 8x8 blocks 22a-22at, which respec- 

levels, an image may be compressed before being sent over lively correspond to the 8x8 blocks A-D of pixels in the 

the internet. Or, to increase the image-storage capacity of a macro block 10. Thus, still referring to FIGS. IB-ID, six 

CD-ROM or server, an image may be compressed before 8x8 blocks of pre-compression pixel data are generated fiir 

being stored thereon. each macro block 10: four 8x8 blocks 22a-22ii of pre- 

Such image compression typically involves reducing the compression Y values, one 8x8 block 18 of pre-compression 

number of data bits necessary to represent an image. C^ values, and one 8x8 block 20 of pre-compression C/^ 

Unfortunately, many compression techniques are lossy. That ^ values. 

is, visual information contained in the original image may be FIG. 2 is a general block diagram of an MPEG compres- 

lost during compression. This loss of information may cause sor 30, which is more commonly called an encoder 30. 

noticeable differences, often called visual artifacts, in the Generally, the encoder 30 converts the pre-compression data 

reconstructed image. In many cases, these artifacts are for a firame or sequence of frames into encoded data that 

undesirable, and thus significantly reduce the visual quality 45 represent the same frame or frames with significantly fewer 

of the reconstructed image as compared to the quality of the data bits than the pre-compression data. To perform this 

original image. conversion, the encoder 30 reduces or eliminates redundao- 

Referring to FIGS. 1-3, the basics of the popular block- cies in the pre-compression data and reformats the remain- 
based Moving Pictures Experts Group (MPEG) compression ing data using eflBdent transform and coding techniques, 
standards, which include MPEG-1 and MPEG-2, are dis- 50 More specifically, the encoder 30 includes a frame- 
cussed. For purposes of illustration, the discussion is based reorder btiffer 32, which receives the pre-compression data 
on using an MPEG 4:2:0 fomaat to compress images rep- for a sequence of one or more frames and reorders the 
resented in a Y, C^ C^,, color space, although the basic frames in an appropriate sequence for encoding. Thus, the 
concepts discussed also apply to other MPEG formats and reordered sequence is often different than the sequence in 
images represented in other color spaces, and to other 55 which the frames are generated. The encoder 30 assigns each 
block-based compression standards such as the Joint Pho- of the stored frames to a respective group, called a Group Of 
tographic Experts Group (JPEG) standard, which is often Pictures (GOP), and labels each frame as either an intra (I) 
used to compress still images. Furthermore, although many frame or a non-intra (non-I) frame. The encoder 30 always 
details of the MPEG standards and the Y, C^, C^ color space encodes an I-frame without reference to another frame, but 
are omitted for brevity, these details are well-known and are 50 can and often does encode a non-I frame with reference to 
disclosed in a large number of available references. one or more of the other frames in the GOP. The encoder 30 

Referring to FIGS. 1-3, the MPEG standards are often does not, however, encode a non-I frame with reference to 

used to compress temporal sequences of images — which are a frame in a different GOP. 

also called video frames — such as found in a television Duringtheencodingof an I frame, the 8x8 blocks (FIGS, 

broadcast. Each video frame is divided into areas called 65 IB-ID) of the pre-compression Y, C^, and C^ values that 

macro blocks, which each include one or more pixels. FIG. represent the I frame pass through a summer 34 to a Discrete 

lA is a 16-pixel-by-16-pixel macro block 10 having 256 Cosine TransfiDrm (DCI) circuit 36, which transfonaas these 
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blocks of values into respective 8x8 blocks of one DC follows the horse, then the background exhibits relative 

coeffident and sixty-three AC coefficients. That is, the motion. Generally, each of the succeeding frames in which 

summer 34 is not needed when the encoder 30 encodes an the object appears contains at least some of the same macro 

I frame, and thus the pre-compression values pass through blodcs of pixels as the preceding frames. But such matching 

the summer 34 without being summed with any other values. 5 macro blocks in the succeeding frame often occupy respec- 

As discussed below, however, the summer 34 is often tive frame locations that are di£ferent than the respective 

needed when tbc encoder 30 encodes a non-I frame. A frame locations they occupy in the preceding frames, 

quantizer 38 limits each of the coefficients to a respective Alternatively, a macro block that* includes a portion of a 

maximimi value, and provides the quantized AC (nonzero stationary object (e.g., tree) or background scene (e.g., sky) 

frequency) and DC (zero freqtiency) coeflBcients on respec- may occupy the same frame location in a succession of 

tive paths 40 and 42. A predictive encoder 44 predictively frames. In either case, instead of encoding each frame 

encodes the DC coefficients, and a variable-length coder 46 independently, it takes fewer data bits to say "locations X 

converts the quantizedACcoefBcients and the quantized and and Z of frame #1 (non-I frame) contain the same macro 

predictively encoded DC coefficients into variable-length ^^^^ locations S and T, respectively, of frame #0 

codes, such as Huffman codes. These codes form the ^ 0 frame)." This "statement" is encoded as a motion vector, 

encoded data that represent the pixel vahtes of the encoded ^or a stationary or relatively slow-moving object or back- 

I frame. A transmit buffer 48 then temporarily stores these ^ "^^"^^^ '^^^^^ 

to zero 

codes to allow synchronized transmission of the encoded ^' 

data to a decoder (discussed below in conjunction with FIG. "^"f*^ sp«nflcaUy and stm refernng to FIG. ^during the 

3). Alternatively, if the encoded data is to be stored instead ,„ f °^ t""";' 7°'"° P^-^^'^" ^ """P^ 

J ^ J ' LI 1 20 the pre-compression Y values (the Co and Co values arc not 

of transmitted, the coder 46 may provide the vanable-length ^^j^g'^otion prediction) of mfcro bloAs in the non-I 

codes direcuy to a storage medium such as a CU-KOM. ^^^^ ^ ^^^^ y values of macro blocks in the 

If the I frame will be used as a reference (as it often wiU reference frame to identify matching macro blocks. For each 

be) for one or more non-I frames in the GOP, then, for the macro block in tbc non-I frame for which a match is found 

following reasons, the encoder 30 generates a corresponding 25 in the reference frame, a motion predictor 58 generates a 

reference frame by decoding the encoded I frame with a motion vector that specifies the location of the matching 

decoding technique that is similar or identical to the dccod- macro block in the reference frame. Thus, as discussed 

ing technique used by the decoder (FIG. 3). When decoding below in conjunction with FIG. 3, dtuing decoding of these 

non-I frames that are referenced to the I frame, the decoder macro blocks of the non-I frame, the decoder uses the 

has no option but to use the decoded I frame as a reference 3Q motion vectors to obtain the pixel values for these macro 

frame. Because MPEG encoding and decoding are lossy, the blocks from the matdiing macro blocks in the reference 

pixel values of the decoded I frame wiU often be different frame. The predictive encoder predictively encodes the 

than the pre-compression pixel values of the I frame. motion vectors, and the coder 46 generates codes for the 

Therefore, using the pre-compression I frame as a reference predictively encoded motion vectors and provides them to 

frame during encoding may cause additional differences in 35 the transmit buffer 48. 

the decoded non-l frame because the reference frame used Furthermore, because a macro block in the non-I frame 

for decoding (decoded I frame) would be different than the and a matching macro block in the reference frame are often 

reference frame used for encoding (pre-compression I similar but not identical, the encoder 30 encodes these 

frame). differences along the with motion vector so the decoder can 

Therefore, to generate a reference frame for encoding that 40 account for them. More specifically, the motion predictor 58 

will be similar to or the same as the reference frame used for provides the decoded Y values of the matching macro block 

decoding, the encoder 30 includes a dequantizer 50 and an of the reference frame to the summer 34, which effectively 

inverse DCT circuit 52, which are designed to mimic the subtracts, on a pixel-by-pixel basis, these Y values from the 

dequantizer and inverse DCT circuit of the decoder (FIG. 3). pre-compression Y values of the matching macro block of 

The dequantizer 50 dequantizes the quantized DCT coeffi- 45 the non-I frame. These differences, which are called 

cients from the quantizer 38, and the circuit 52 transforms residuals, are arranged in 8x8 blocks and are processed by 

the dequantized DCT coefficients back into corresponding the DCT circuit 36, the quantizer 38, the coder 46, and the 

8x8 blocks of Y, C^, and C^ pixel values. Because of the btiffer 48 in a manner similar to that discussed above, except 

losses incurred during quantization and dequantization, that the quantized DC coefficients of the residual blocks are 

however, some or all of these decoded pixel values may be 50 not predictively encoded by the predictive encoder 44. 

respectively different than the corresponding pre- Additionally, it is possible to use a non-I frame as a 

compression pixel values. These decoded pixel values then reference frame. When the non-I frame will be used as a 

pass through a summer 54 (used when generating a refer- reference frame, the quantized residuals from the quantizer 

ence frame from a non-I frame as discussed below) to a 38 are respectively dequantized and inverse transformed by 

reference-frame buffer 56, which stores the reference frame. 55 the dequantizer 50 and the inverse DCT circuit 52 so that this 

During the encoding of a non-I frame, the encoder 30 non-I reference frame will be the same as the one used by the 

initially encodes each macro-block of the non-I frame in at decoder for the reasons discussed above. The motion pre- 

least two ways: in the manner discussed above for I frames, dictor 58 provides the decoded Y values of the reference I 

and using motion prediction, which is discussed below. The frame from which the residuals were generated to the 

encoder 30 then saves and transmits the resulting code 60 summer 54, which adds the respective residuals from the 

having the fewest bits. This technique insures that the macro circuit 52 to these decoded Y values of the reference I frame 

blocks of the non-l frames are always encoded using the to generate the respective Y values of the reference non-I 

fewest bits. frame. The reference-frame buffer 56 then stores the refer- 

Wth respect to motion prediction, an object in a frame ence non-I fi^me along with the reference I frame for use in 

exhibits motion if its relative position changes in the sue- 65 encoding subsequent non-I frames. 

ceeding frames. For example, a horse exhibits relative Still referring to FIG. 2, the encoder 30 also includes a 

motion if it gallops across the screen. Or, if the camera rate controller 60 to insure that the transmit buffer 48, which 
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typically transmits the encoded firame data at a fixed rate, N. Ngan, D. W. Lin, and M. L. Liou, "Enhancement of 

never overflows or empties, Le., underflows. If either of Image Quality for Low Bit Rate Video Coding,** IEEE 

these conditions occurs, errors may be introduced into the Transactions on Circuits and Systems, Vol. 38, No. 10, 

encoded data. For example, if the buffer 48 overflows, data October 1991, pp. 1221-1225, disclose deblocking tech- 
from the coder 46 is bst Thus, the rate controller 60 uses 5 niques that are implemented during image encoding. But 

feed back to adjust the quantization scaling factors used by most images and video sources are encoded according to 

the quantizer 38 based on the degree of fullness of the internationally agreed-upon compression standards such as 

transmit buffer 48. The more full the buffer 48, the larger the MPEG, so altering the encoding algorithms is impractical if 

controller 60 makes the scale factors, and the fewer data bits not impossible if one wishes to design an encoding system 
the quantizer 40 generates. Conversely, the more empty the lO that complies with one or more of these standards, 

buffer 48, the smaller the controller 60 makes the scale Other references, inchiding T. O'Rourke, R. Stevenson, 

factors, and the more data bits the quantizer 40 generates. "Improved Image Decompression for Reduced Transform 

This continuous adjustment insures that the buffer 48 neither Coding Artifacts," IEEE Transactions On Circuits And Sys- 

overflows nor underflows. tems For Video Technologies, Vol. 5, No. 6, December 1995, 

FIG. 3 is a block diagram of a conventional MPEG and Y. Yang et al, "Projection-Based Spatially Adaptive 

decompressor 60, which is more commonly called a decoder Reconstruction of Block-Transform Compressed Images,'* 

60 and which can decode frames that are encoded by the IEEE Transactions on Image Processing, Vol. 4, No. 7, July 

encoder 30 of FIG. 2. 1995, disclose deblocking techniques that are implemented 

For I frames and macro blocks of non-I frames that arc not during image decoding. For example, O'Rourkc et al. 

motion predicted, a variable-length decoder 62 decodes the describe a statistical discontinuity-preserved image model 

variable-length codes received from the encoder 30. A ^ statistical image compression model, and a technique 

prediction decoder 64 decodes the predictively encoded DC ^^r generating maximum a posteriori (MAP) estimations of 

coefficients, and a dequantizer 65, which is similar or boundary pixels given based on these two models. O'Rourke 

identical to the dequantizer 50 of FIG. 2, dequantizes the estimates the values of the boundary pixels by itera- 
decoded AC and DC coefiBcients. An inverse DCT circuit 66, ^ lively solving a convex constrained optimization problem, 

which is similar or identical to the inverse DCT circuit 52 of Similarly, the Yang reference assumes that changes in neigh- 

FIG. 2, transforms the dequantized coefficients into pixel ^onng, pixel values, i.e., the vahies of pixels on either side 

values. The decoded pixel values pass dirough a summer 68 ^f a boundary, should be at a minimum, and then, like 

(which is used during the decoding of motion-predicted O'Rourke, proceeds to estimate the values of the boundary 

macro blocks of non-I frames as discussed below) into a P™U by iteratively solving the convex constrained optimi- 

frame-ieorder buffer 70, which stores the decoded frames z^^io^ problem. But such techniques often require too much 

and arranges them in a proper order for display on a video computation time for implementation in a real-time system, 

display unit 72. If the I frame is used as a reference frame, AdditionaUy, such techniques often operate on boundaries 

it is also stored in the reference-frame buffer 74. that are not blocky. Unfortunately, when such techniques are 

For motion-predicted macro blocks of non-I frames, the to boundaries that are not blocky the quality of the 

decoder 62, dequantizer 65, and inverse DCT 66 process the "°age may be de^aded because generaUy, the assumpUon 

residuals as discussed above. The prediction decoder 64 ^yj^^ techniques is that the difference between a 

decodes the motion vectors, and a motion interpolator 76 P^^^ neighbonng pixels should be small. Although 

provides to the summer 68 the pixel values from the macro ^ l^^*' assumption is correct some of the time, it is 

blocks in the reference frame that the motion vectors point frcquendy incorrect, particularly m areas of an image mclud- 

to. The summer 68 adds these reference pixel values to the cd^cs. 

residuals to generate the pixel values of the decoded macro ^tiU other references describe deblocking techniques that 

blocks, and provides these decoded pixel values to the employ low-pass filters along the block boundaries, 

frame-reorder buffer 70. If the non-I frame is used as a Unfortunately, such low-pass filtering may lead to blurring 

reference frame, it is stored in the reference-frame buffer 74. ^^^^^^ boundaries. Some of these techniques, such as 

AmoredetaileddiscussionoftheMPEGencoder30and ^^^^ ^ Ramamurthi and A. Geisho, "NonUnear 

decoder 60 of FIGS. 2 and 3, respectively, is avaflablc in f^^^^^"^^"^' Post-processmg of Bl^k Coded Images, 

many publications inchiding "Video Compression" by Peter ^^^^ ^'''^'i'f'^.cci>t!^ ^?no?^ 

D.Symes, McGraw-Hill, 1998. Furthermore, there are other ,0 ^^^f.^^l' ^^^'^^ ^ Pf" 

11 1™ ui 1 u J • . u • r J 1258-1268, attempt to avoid blurrme the boundanes by 

well-known block-based compression techmques for encod- . . ' ^ ^ ^ , ^. , . , . . , 

ing and decoding images estimatmg the vahies of the boundary pixels id the ongmal 

V. r • .t -1 .t 1 . , image and then adaptively choosing different types of fillers 

Refemng to FIG. lA, a Problem with block-based com- ^ ^ sharpness of the boundaries in the original 
picssion techniqi«s such as the MPEG standard is that the ^ Unfortunately, accurately estimating original bound- 
loss of visual i.^nnaUon during wmpresswn may cause 55 ^ly vahies from a highly coiipressed image may be very 
some or all of the .elective boundanes between the 8x8 ^^^^ ^ „j ^ ^^^^ ■ ^ 
pixel blocks A-D and between conUguous macro blocks 10 inadequate for accurate boundary-value estimation, 
to be nouceable to a viewer More q)eciflcally. the compres- Furthermore, like some of the techniques described above, 
SK.n lossM may cause an abrupt chimge m the pixel values techniques often operate on all of the boundaries in an 
across a boundary, thus inakmg the boundary visible. Such «, in,age whether they are blocky or not. and thus may unnec- 
a v^ible boundary is often desmbed as block/ or as ^ ^j,^ ^ ^ ^ ^ 
exhibiting a -blodqT- artifact, and the process of reducing computationally intensive for many apphcations. 
the seventy of blocky artifacts. I.e., making blocky bound- 
aries invisible to a viewer is often called deblocking. SUMMARY OF THE INVENTION 

Some references, including C. Reeve and J. S. Lim, 65 In one aspect of the invention, an image processing circuit 

"Reduction of Blocking Effects in Image Coding," Optical includes a processor that receives a portion of an image that 

Engineering, Vol. 23, No. 1, Jan/Feb 1984, pp. 34-37, and, includes a boundary and first and second contiguous pixels 
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disposed on opposite sides of the boundary, the first and 
second pixels having respective first and second pixel val- 
ues. The processor generates a boundary value &Dm the first 
and second pixel values, compares the boundary value to a 
comparison value, and reduces the difference between the 
first and second pixel values if the boundary value has a 
specified relationship to the comparison value. 

Because such a processing circuit operates on an image 
after it has been decoded, it does not change the way an 
image is encoded or decoded, and thus is compatible with all 
block-based compression standards. Furthermore, the com- 
parison value can be set so that the processing circuit 
operates only on blocky boundaries, aiKl thus does not 
degrade boundaries that are not blocky. Additionally, the 
processing circuit can operate on a sequence of video frames 
in real time. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. lA is a conventional macro block of pixels in an 
image. 

FIG. IB is a conventional block of pre-compression Y 
values that respectively correspond to the pixels in the 
macro block of FIG. lA. 

FIG. IC is a conventional block of pre-compression 
values that respectively correspond to the pixel groups in the 
macro block of FIG. lA. 

FIG. ID is a conventional block of pre-compression 
values that respectively correspond to the pixel groups in the 
macro block of FIG. lA. 

FIG. 2 is a block diagram of a conventional MPEG 
encoder. 

FIG. 3 is a block diagram of a conventional MPEG 
decoder. 

FIG. 4 is a schematic block diagram of an embodiment of 
an imaging processing circuit according to the invention. 

FIG. 5 is a flow chart that explains the operation of the 
image processing circuit of FIG. 4. 

FIG. 6 is a detailed view of a macro block having 
boundaries that the image processing circuit of FIG. 4 
operates on. 

FIG. 7A is a functional block diagram of an embodiment 
of a filter that reduces the differences in pixel values across 
vertical image boundaries according to the invention. 

FIG. 7B is a functional block diagram of an embodiment 
of a filter that reduces the differences in pixel values across 
horizontal image boimdaries according to the invention. 

FIG. 8A is a functional block diagram of another embodi- 
ment of a filter that reduces the differences in pixel values 
across vertical image boundaries according to the invention. 

FIG. 8 B is a functional block diagram of another embodi- 
ment of a filter that reduces the differences in pixel values 
across horizontal image boundaries according to the inven- 
tion. 

DETAILED DESCRIPTION OF THE 
INVENTION 

FIG. 4 is a schematic block diagram of an image pro- 
cessing and display circuit 80, which includes an image 
processing circuit 82 and an image display circuit 84. The 
circuit 80 may be used to process and display individual 
images or a sequence of video frames. The image processing 
circuit 82 includes a conventional storage circuit 86 for 
storing image data received from a decoder such as the 
decoder 60 of FIG. 3. The circuit 82 also includes an image 
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processor 88, which in one embodiment includes conven- 
tional hardware components (not shown) and which reduces 
the pixel differences across bbcky image boundaries as 
discussed below. In one embodiment of the invention, the 

5 storage circuit 86 is part of the decoder. For example, the 
storage circuit 86 may be the firamc-rcordcr buffer 70 of FIG. 
3. In another embodiment, the storage circuit 86 is part of the 
processor 88. In yet another embodiment, the processing 
circuit 82 does not include the storage circuit 86, and the 

10 processor 88 receives the image data directly from the 
decoder. The display circuit 84 includes an image storage 
circuit 90, which stores the processed image data from the 
processor 88, and includes a display device 92, which 
displays the images stored in the circuit 90. 

15 FIG. 5 is a flow chart that shows the general operation of 
one embodiment of the image processor 88 of FIG. 4. For 
example purposes, the operation of the processor 88 is 
discussed with respect to the pixel Y values, it being 
understood that the operation is the same for the pixel C^ 

^0 and C/{ values and for the luminance and chroma values of 
other color spaces. 

Id step 100, the processor 88 first calculates a threshold 
value based on the decoded values of some or all of the 
pixels in two contiguous pixel blocks that share a boundary. 

^ In one embodiment, the pixel blodcs arc 8x8, it being 
understood that the pixel blocks may have dimensions other 
than 8x8. 

Next, in step 102, the processor 88 calculates a boundary 
value from the values of the pixels that lie along the 
boundary. Although the processor 88 is described as per- 
forming step 100 before step 102, the order of these steps 
can be reversed. 

Then, in step 104, the processor 88 compares the thresh- 
old value to the botmdary value. 

In step 106, if the boundary value differs from the 
threshold value in a predetermined manner (e.g., is greater 
than, is less than), then the processor 88 identifies the 
boundary as blocky, and in step 107, filters the pixel values 
^ that lie along the botmdary. In one embodiment, the proces- 
sor 88 filters the pixel values by reducing the differences 
between the values of pixels on one side of the boundary and 
the values of pixels on the other side of the boundary. If the 
boundary value does not differ fi-om the threshold value in 
^5 the predetermined way (c.g., is not greater than, is not less 
than), then the processor 88 identifies the boundary as not 
blodcy. 

Referring to step 108, the processor 88 then processes all 
of the remaining boundaries of the image in this maimer. In 

5Q one embodiment, the processor 88 starts in the upper left 
comer of the image and works toward the lower right comer. 

Referring to step 110, after the processor 88 processes all 
of the boundaries with respect to the Y values, the processor 
88 repeats steps 100-108 for the Cj, and C^ values. 

55 Referring to step 112, after the processor 88 processes aU 
the boundaries in the image with respect to the C^ and Cj, 
values, the processor 88 repeats steps 100-110 for any 
remaining images. 
Therefore, because the processor 88 operates on the 

60 boundaries of an image after the image has been decoded, 
the described de-bloddng technique neither needs a change 
in nor makes a change to the techniques used to encode and 
decode the image, and thus is compatible with any block- 
based compression standard. Furthermore, as discussed 

65 below, because the calculations of the threshold and bound- 
ary values and the filtering algorithm are relatively simple, 
the processor 88 can operate on the boundaries fast enough 
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for real-time applications such as the enhaDcement of HDTV there are no chroma-difference boundaries within a macro 
video firames. Additionally, experimental observation of blodc, and thus the processor 88 operates only on the 
images compressed using standards such as ISOyiTU, boundaries between macro blocks with respect to the and 
MPEG, and H.263 indicates that a significant number of values. The processor 88, however, operates on these 
boimdaries are not blocky, and that the decoded image is 5 boundaries with respect to the C^^ and Cq values in a manner 
often of the best visual quaUty when these **unblock/' ^^^^ ^o that described below for the Y values. In other 
boundaries arc left unaltered. Thus, because the processor 88 formats, however, there may be chromaKiiffercncc bound- 
filters only the boundaries it identifies as blocky. it does not ^ries within a macro block, in which case the processor 88 
degrade the quaUty of the image by filtenng the boundaries operates on these boundaries too. Furthermore, aUhough the 
that are not blocky. 10 ^^^^ ^lock 120 is described as being 16x16 and the blocks 
Referring to FIGS. 6, 7A, and 7B, the operation of the a-D are described as being 8x8, the macro block 120 and 
image processor 88 of FIG. 4 according to an embodiment the blocks A-D may have other dimensions, 
of the invention is discussed. Generally, to determine « ,1 ^ . , , . i. . 
whether or not a boundary between two contiguous pixel StiU referring to Fia 6. the operation of the processor 88 
blocks is blocky, the processor 88 compares the respective „ embodiment of the mvenUon b discussed 
average roughnesses inside the two blocks to the roughness " =^'TP^'= P""}!??^. ! ^ '"'^ 
at the boundary. That is, the boundary value (step 102 of boundaries L22 and 124 and the Y values is discussed. 
FIG. 5) is a fiinction of the roughness at the boundary " ''"?g "nderstood that the operation is smnlar with respect 
between the two btocks, and the threshold value (step 104 ^. Z^"^ boundanes-whether between the blocks A-D 
no. 5) is a function of the respective average ro^hnesscs ^ i^'^" 1,^^"° ^" between conUguous macro 

t wi I jc *u DO A **c blodcs 120— or with respect to the C« and Co values. 

inside the two blocks. If the processor 88 identifies a ^ , , „. , , „ ^ . • 

boundary as being blocky. then it implements the conven- ^J^'T'^l'^/T ^ !f ^^"""'^ 

tional finite-impuke-response (FIR) filters of HGS. 7A and ^^"'^ ^ ^ 

7B to "smoothen" the differences between the values of the In this embodiment, the horizontal roughness of a block 

pixels that lie along and on opposite sides of the boundary. 25 respect to the Y values equals the average horizontal 

Thus, this embodiment exploits the experimental observa- difference between the Y values of horizontally adjacent 

tion that in a typical real- world image before compression, pixels within the block. (Similarly, for the 4:2:0 MPEG 

the roughness within a particular block is the same as or is format discussed in conjunction with FIGS. 1-3, the hori- 

close to the respective roughnesses at its boundaries with zontal roughnesses with respect to the Q, and values 

adjacent blocks. But for reasons discussed above, lossy 30 equal the average horizontal differences between the C/t and 

compression schemes may cause a block to have a different values, respectively, of adjacent pixel groups, like the 

(typically higher) degree of roughness along its boundaries groups 14 of FIG. lA, within the block.) Likewise, the 

iian within itself, and this difference in roughness is visible vertical roughness of a block with respect to the Y values is 

as a blocky boundary. Furthermore, although this embodi- equal to the average vertical difference between the Y values 

ment is described below with respect to the Y values, it can 35 of vertically adjacent pixels within the block. (Similarly, for 

be used with the and values and the luminance and the 4:2:0 MPEG format discussed in conjunction with FIGS, 

chroma values of other color spaces. 1-3, the vertical roughnesses with respect to the and 

FIG. 6 is a 16x16 macro block 120, which includes four values equal the average vertical differences between the 

8x8 pixel blocks A-D. Although shown in more detail, the and values, respectively, of adjacent pixel groups.) For 

macro block 120 and the blocks A-D are similar to the 40 example, the respective horizontal roughnesses R^a and R^a 

macro block 10 and the blocks A-D of FIG. LA. Each of the of the blocks A and B of BG. 6, respectively, are represented 

pixels in the respective blocks A-D has a respective row and by the following formulas: 
column location. For example, the pixel in the block A 

is located at the intersection of row 2 and column 4 of the 1 ^ ^ 

block A. A similar coordinate scheme is used for the blocks 45 '''"^56^^''**'" 
B-D. Furthermore, four boundaries are respectively located 

between the blocks A-D. For example, a boundary 122 is 1 ' ' (-^^ 

located between the blocks A and B, and a boundary 124 is '^^^ " £ '^j*^ 

located between the blocks A and C. The widths of the ^ 

boundaries 122 and 124 are exaggerated for illustration, but 50 

in actuality, the pixels in column 7 of block Aare contiguous ^^ere a,^ is the Y vahie of the pixel in row y and column 

with the reacti ve pixels m cohunn 0 of block B, and the ^ of blodc A, and b„ is the Y value of the pixel in row y and 

pixels m row 7 of block A are contiguous with the respective column x of block B 

pixels in row 0 of block C. For example, referring to the m i .1. ^- . l ^ jr. ^ .i. 

r J *L - 1 -.1. • iL Similarly, the vertical roughnesses R^„ and R^ of the 

boundary 122, the pixel ao 7 IS contiguous with the pixel bn o, 55 . 1 * . ^ 1 . j 1. .urn 

- ; - ' -^L JiT • 1 1_ J ' blodcs A and C, respectively, are represented by the follow- 

the pixel aj 7 is contiguous with the pixel b^ q. and so on. ^ formulas: 

As discussed in conjunction with FIGS. 1-3, because the °™ ^ 

DOT transform is performed on 8x8 blocks of pixel values, ^ ^ 

the macro block 120 is divided into the 8x8 blocks A-D. f>»v = — X Z '^'/-m -«j 

Therefore, for example purposes, the processor 88 is 60 ^^j=i r=o 

described as operating on the internal boundaries between 

the blocks A-D with respect to the Y values, it being /j^, _ i. V V |^ ^ _ ^ | 

understood that the processor 88 operates on the external 56^ ^ 
boundaries between macro blocks with respect to the Y 

values in a similar fashion. But because in the MPEG 4:2:0 65 

format each macro block has associated therewith only one where c^^ i 

8x8 block of C« values and one 8x8 block of values, x of the block C. 
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Furthermore, the horizontal boundary roughness of R^^^ to a comparison value to determine whether or not 

the boundary 122 between the blocks A and B is represented the boundary 122 is blocky. In this embodiment, is 

by the following formula: calculated from equation (11) as: 

jr=0 

Thus, equals the average difference between the Y Therefore, in this embodiment, the boundary 122 is blocky 

values of the respective pairs of contiguous pixels in colimio jf ; 
7 of block A and colunm 0 of block B. lO 



Likewise, the vertical boundary roughness R^^, of the 



boimdary 124 between the blocks A aiKl C is represented by ff^^ > -[(/tAh + Rah) + TkiPA + Pa) 

the following formula: ^ 



1 (13) 



[6) Thus, the processor 88 identifies the boundary 122 as 

~^oJ being blocky and filters this boundary as discussed below 

only if the boundary value, here the horizontal-boundary 

Tlius, R^c equals the average difference between the Y ^^g^^.f ss R^ is greater than the comparison value, 

values of the respective pairs of contiguous pixels in row 7 ^ Similarly, the processor 88 idenufies the boundary 124 

of block A and row 0 of block C. 20 ^^^"^^^ ^^^^ A and C as blocky and filters this 

It has been experimentally determined that the horizontal boundary when the following equation is true: 
boimdary 122 is blocky ff the horizontal boundary roughness 

exceeds the average of the horizontal roughnesses R^ o ^ o o ^* » n 

and R^;, by a first predetermined amount, and that the /Fac- 2(/?>.v + /?cv)> yCP. + Pc) 
boundary 124 is blociy if the vertical boundary roughness 

R^c exceeds the average of the vertical roughnesses R^„ and ^ , rr. • * ...lu 

R^ by a second prectetermined amount. To carry o^t this ^^^^^'^ f an cxpcnmcntally dctcnnincd vertical threshold 

calculation, the processor 88 calculates the average Y values constant In one aspect of this embodiment, T,^.04. 

Pa> Pb. Pc for the pixels in blocks A, B, and C, respectively, ^ equation (11), it is convenient to rcwnte cquaUon 

according to the following foraiulas: (15) again so that one can compare a boundary value, here 

30 the boundary roughness R^, to a comparison value M^^^ to 

J 7 7 (7) determine whether or not the boundary 124 is blocky. In this 

= ^2 ^ Ojyr embodiment, M^^ is calculated from equation (14) as: 



(9) Therefore, in this embodiment, the boundary 124 is blocky 
if: 



40 

The processor 88 identifies the boundary 122 between the Rac>~ URav •»- Rcv) + T^iPA + Pc)] 



blocks A and B as blocky when the following equation is 
true: 



2'' 



Although in this embodiment the respective boundary 
-(^Ah values R^ and R^^- and comparison values and M^^- 

2 ^ — ^ ^ 7a functions of the Y pixel values in two adjacent blocks of 

Pa + Pb Pa + Pb 2 image, in other embodiments these values may be 

where T^ is an experimenuUy determined threshoki con- ^^^^l^^^ ot other data such as the pixel values in nonadja- 
stanl. In one aspccTof this embodiment, T,=0.05. More centblocksmthe same frame, pixel values m blocks of other 
spedficaUy, because the human visual system is more sen- ^^"^^^ ^ ""'^^ sequence, motion-vector values, or 
sitive to differences in roughness than the to the actual transform-domam values such as DCT coefBcienls. 
degree of roughness, the processor 88 filters the boundary Next, to reduce the undesirable effects that blocky bound- 
122 only if the average of the horizontal roughnesses R^a °° ^ image, the processor 88 implements a filter, 

and R^;, differs from the average of the horizontal-boundary such as a FIR filter, to smoothen the boundaries that are 
roughness R^ by more than T,, divided by 2. The equation 55 identified as being blocky. In this embodiment, the filter 
10 can be rewritten as: operates on the Y values of the pixels that are contiguous 

with the blocky boundary. For example, if the processor 88 
1 Ta (11) identifies the boundary 122 as being blocky, then it filters the 

Pab - ^(Ram + Rbm) >YiPA + Pb) y values of the 16 pixels— the eight pixels in cohimn 7 of 

60 block A and the eight pixels in column 0 of block B — that 
are contiguous with the boundary 122. Likewise, if the 
processor 88 identifies the boundary 124 as being blocky, 
Rab - 2('?Ah ■»- PBh) ^ yC^A + Pb) then it filters the Y values of the 16 pixels — the eight pixels 

in row 7 of block A and the eight pixels in row 0 of block 
65 C — that are contiguous with the boundary 124. 
It is convenient to rewrite the equation again so that one More specifically, referring to FIGS. 7A and 7B, in this 
can compare a boundary value, here the boundary roughness embodiment, the processor 88 implements a filter that sets 
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ihe Y value of a filtered pixel equal to a function of its Y contiguous pixel in block B or vice-versa, has no affect on 

value and the Y values of the two contiguous pixels on either the result of the filtering. Thus, the pixels a^^-a^ ^ and 

side of the filtered pixel in a direction perpendicular to the ^0,0-^7,0 can be filtered in any order. Also, because the 

boundary. pre-filtered Y values of a^ y and bj q, and not the filtered Y 

Referring to FIG. 7A, for a horizontal boundary such as 5 values of sl^j and bg q, are used during the filtering of ag 

the boundary U2, the Y value of the filtered pixel hpl is set the filtered Y value of a^ ., is not equal to the average of the 

equal to a function of its Y value and the Y values of the two ^ ^^^^^ and the filtered Y values of 83 7 and b^^Q, 

horizontally contiguous pixels hpO and hp2. For example, to "^^"^^ ^ ^^^^ aPP«" ^ decoded image after deblock- 

filter the Y vahie of a boundary pixel (hpl), the mg. Likewise, the filtered Y value of bj^o is not equal to the 

processor 88 sets the Y vahie of the pixel ag^, equal to a 10 averageof the Y value of ba^^ and the filtered Y values of 33^7 

function of the Y values of the pixels ^ ^ (hpO), aj 7 (hpl), ^^.o- 

and b3 0 (hp2). The processor 88 then filters the Y values of Similarly, refemng to HGS. 6 and 7B, the filtermg of the 

all of the other pixels in column 7 of block A and column 0 ^ values of the pixel and the vertically adjoining pixel 

of block B in a simUar manner to effectively reduce the ^ discussed in conjunction with the smoothing of the 

respective differences between the Y values of the pixels in 15 boundary 124 according to an embodiment of the invention, 

column 7 of block Aand the Yvaluesof the respective pixels processor 88 calculates the average pixel value far , 

in column 0 of block B. That is. the processor 88 smoothens P^^^ ^e^' a7;3.and Co^ (vpO, vpl, and vp2). Then, the 

the boundary 122 by reducing the average difference processor 88 stores this resulting filtered pixel value for a7^ 

between the Y values of the pixels a7o . . . 7 and the Y temporary memory location. Next, using the pre-filtcred 

values of the respective pixels boo • • bo 7. ' 20 P^^^^ ^^^^^ P^^^ ^,3' processor 88 sets the value 

SimUarly, referring to HO. 7B, for a vertical boundary pixel Cq^ (vpl) equal to the average pixel value for the 
sucb as the boundary 124, the Y vahie of the filtered pixel P^^^ *7^> Co^> and c^^ (vpO, vpl. and vp2). Therefore, for 
vpl is set equal to a function of its Y value and the Y values ^® reasons stated above, this embodiment allows the pro- 
of the two vertically contiguous pixels vpO and vp2. For cessor 88 to filter the pixels 37 0-87 7 and Co,o-Co,7 in any 
example, to filter the Y value of a boundary pixel Co 4 (vpl), 25 ^^^^^'^ ^^^^^ ^f is not equal to the 
the processor 88 sets the Y value of the pixel Cq 4 equal to a average of the value of a^ 3 and the filtered vahies of a^^ and 
function of the Y values of the pixels a^ 4, (vpO); Cq 4. (vpl). ^o^> ^nd the filtered value of Co,3 is not equal to the average 
and Cj 4, (vp2). The processor 88 then filters the Y values of ^^^^^ ^1,3 and the filtered values of 37^ and Cq^. 
all of die other pixels in row 7 of block A and row 0 of block Referring to FIGS. 6. 8A, and 8B, in another embodiment 
C in a similar manner to effectively reduce the respective 30 invention, the processor 88 uses different roughness 
differences between the Y vahies of the pixels in row 7 of and filtering calculations to respectively identify and 
block A and the Y values of the respective pixels in row 0 smoothen blodcy boundaries. One difference between this 
of block C. That is, the processor 88 smoothens the bound- embodiment and the previous embodiment discussed above 
ary 124 by reducing the average difference between the Y ^ conjunction with FIGS. 7A and 7B is that in this 
values of the pixels 7 . . - a7 7 and the Y vahies of the 35 embodiment, the processor 88 is often better able to distin- 
respective pixels Cqo - ' • Cq?. ' between a sharp edge of an object and a bbcky 

In one aspect of 'this embodiment, the processor imple- boundary. Furthermore, like the previous embodiment, 
ments an averaging filter that sets the Y value of the filtered although this embodiment is discussed with respect to the Y 
pixel equal to the average of its pre-filtered Y value and the ^^^^ boundaries within a macro bbck. the pro- 
Y values of the contiguous pixels. For example, in this 40 ^^^^ operates on the Cr and Cj^ values and the bound- 
embodiment, referring to HGS. 7A and 7B. hpl^^^ aries between contiguous macro blocks in a simUar manner. 
(Mp.-;i^.«^hp W.«.^*p2^.;^.^/3 and vpl^_^ 1° embodiment, the processor 88 implements a mini- 
(vpO^^-P.„^vp1^^.^,„^vpCa/«^^/3- As discussed mum filter to determme the degree of blockmess at the block 
below, the processor 88 uses the pre-filtered Y values of the boundanes. The minimum filter removes the part of the 
pixels it filters to avoid reaching a result that depends on the 45 roughness value that does not result from compression 
order in which the blocks arc filtered. ConsequenUy, after the quantization errors. For example, the horizontal roughness 
processor 88 filters contiguous pixels on either side of a at the boundary 122 is represented by the following 
boundary, neither pixel has a filtered Y value equal to the equation: 
average of the filtered Y values of the pixels next to it. 

For example, referring to FIGS. 6 and 7A. the filtering of 50 _ ^ v ^^^^ 

the Y values of the pixel 83^7 and the horizontaUy adjoining " 8^ ^^^K^ ' VI. ^9*) 
pixel b3 0 is described in conjunction with the smoothing of 
the boundary 122 according to an embodiment of the 

invention. First, the processor 88 calculates the average Y That is, an element y=i of the summation equals the 
value for the pixels a3 7, and b3 q (hpO, hpl, and hp2). 55 smaller of the absolute value of 7-b^ q ^ values of the 
Then, the processor Si stores this resulting filtered Y value pixels a^ 7 and b^ o* respectively) and T^^, which is either an 
for a3 7 in a temporary memory location such as in an experimentally determined constant or an adaptive variable 
on-board memory array. Next, using the pre-filtered value of related to the quantization cocflSdents used during the 
a3 7. the processor 88 sets the Y value of the pixel b3 0 (hpl) encoding of the image. For example, if T^ is a constant, then 
equal to the average Y value for the pixels aj 7, bj^, and b^^, 60 T^-30 has been found to give good results. Alternatively, if 
(hpO, hpl, and hp2) and stores the resulting filtered Y value it is an adaptive variable, then T^ is determined according 
for b^ Q in another temporary memory location. Then, the to the possible quantization errors. More specifically, it has 
processor 88 puts the filtered Y values for aj 7 and b3 q in the been discovered that a major cause of blocky boimdaries is 
respective memory locations corresponding to the Y values the errors resulting from quantization of the first few coef- 
of these pixels. As stated above, by using the pre-filtered Y 6S ficients of the DCT transform. As discussed above in coo- 
values in all filtering calculations, the order of filtering, i.e., junction with FIG. 2, the quantization circuit 38 quantizes 
fihering a pixel in block A before filtering a horizontally the coefiGcients from the DCT circuit 36 to reduce the 
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number of data bits Deeded to encode a particular pixel Yvaluesoflhe horizontally contiguous pixels in the first two 

block- The quantization circuit 38 applies a respective columns 0 and 1 of block B. 

quantization value to each of the OCT coefficients reprc- similarly, the vertical roughnesses R^, and of the 

senting a pael block. In some apphcations, the quantizer 38 ui ^ a j *• i * j u .u r n 

varies Ihe quantization values from block to block depend- 5 ^lod^s Aand C. respecUvely, are represented by the foltow- 

ing on characteristics such as the degree of visual detail in "^S equations. 

a particular block. In other applications, the quantization 

values are fixed. Therefore, is a fiinction of the first three _ ^ V 

quantization values for each of the two blocks that are ~ ^^lae^-ojj 

contiguous with the boundary in question, and in one aspNSct 

of this embodiment is represented by the following ^ ^ (24) 

equation: ^cr = g 2] l<^o^-<^ul 

_ _ 9ao •»- ^At ^ 9a2 ^^m-*- Hbi + to (18) 
/flf- 2 

15 The processor 88 identifies the boundary 122 as being 
blocky if the following equation is tme: 

where ^^^t^az ^"^b2 first three quantization 

values used during the encoding of blocks A and B, respec- j?^>F MAX(/i^ Rah^T,^ (25) 

tively. 

In a similar manner, the vertical roughness R^^: at 20 Therefore, in this embodiment, the oomparisoD value M^a is 
boundary 124 is represented by the following equation: equal to the right-hand side of equation (25). Furthermore, 

F and T^ are determined e^erimentally, and MAX(R^;^, 

^Ac = - y MiN(ifl7, - fQ,|. r ) example, F-1.4 

~ 8^ ' & . »f Tf,^ have been found to yield good results, although 

^ different values may be used. 



„ ™ . , , , . , J , Likewise, the processor 88 identifies the boundary 124 as 

Here, T-^ IS also either a constant or can be represented by . . / ^„ . . . 

the follov^g equation: ^"""^ ^^"^^^ ^ ^^"^ foUowmg equation is true: 

_ _ qAQ^qAi^qA2^qco^qct^ qc2 (20) 30 ^^>FMAX(R^,)+Ti^ (26) 
2 

Here, the comparison value MAC is equal to the right-hand 
side of equation (26), and F=1.4 and T^,j«8 work well, 

1^ tte"SS>dM 'of W^k^C ^ quantization values used although different values may be used. 

"^^Thf iiLTsS sl r^?S*^ates the horizontal and ver- Referring to RGS. 8A and 8B. in this embodiment the 

ticalroughnesseswithinablockinadifferentmanneMnthe P^°^^ implements a filter that is different from the 

previous embodiment, to identify a blocky boundary, the '^^^^^^J^' .^^ ^^^8. 

processor 88 calculates the horizontal and vertical rough- . specifically the ^cessor 88 implements 

nesscs within a block by respectively averaging the hori- a filter that operates on ^e values of ^e fo^ 

zontal and vertical differences between the valuS of all the ^ to a boundary, two pixeU on either ade of the bou^^ 

horizontally and verticaUy contiguous pixels within the 'T^,^^' f 1!^ V'' ^ where two pixe^ hpO and 

block. But in this embodiment, instead of computing the ^ Tt^fV^ ' ^""Tul 'T^^ 

average horizontal and vertical differences for the values of T ^ f t ^ / ""^S k 

every pbcel in the block, the processor 88 computes these 2L%°u T f u ?l ^'l?^ i ^""""^T" 

average differences only for values of the two aUgnmenls f ^.^^ k^^^ ? / 

(either rows or columns) of pixels adjacent to the block ^n^lf u i 

boundary. Accordingly, the horizontal roughness R^^ of the Lik^^^ to FIG. 86 where pixeU vpO and 

block A is represented by the foUowing eqiution: \^ ^^^^^^^ ^ ^° °/ « honzontal boundary 

^ ^ 132, and two pixels vp2 and vp4 of a second block are on 

50 the other the other side of the boundary 132, the processor 

J? = iy ifl -o I 88 filters the values of vp(M^ based on four functions 

Ah 8^'^** fvO-fv3 of the pre-filtered values of vp0-vp3. 

For example, referring to RGS. 6 and 8A, in one 

™ ^. 11.. fn • • I'c J w . 1 . .u embodiment, if die boundary 122 is blocky, the processor 88 

T1.US, calculat«>n of ^ sin^lified by tatang toe „ ^^^^ ^ '^^^ for y-0.U3, ... .7 

average of the absolute values of the differences between the ^^^j^^ g jf^^ ^y^^ 

Yvaluesof the honzontally contiguous pixels m the last two ^ / o 01 

columns 6 and 7 of block A , ^-1*. 

In a similar manner, the horizontal roughness R^ of block 
B is represented by the following equation: 

where a'^ g corresponds to the filtered Y value of the pixel 

BH-^2^\o,,o b(^^^^^^ and b(y,i3p^^„^ respectively correspond to 

the pre-filtered Y values of the pixels hpO-hp03, i.e., 

65 hpOp^-^/ftrrerf> ^V'^prv-fUtrndy ^^prv/Uurrrdf 

and hp3^„.^^„^of 

Thus, the calculation of R^^ ^ simplified by taking the FIG. 8A. Thus, a'^ g equals the filtered Y value of the pixel 

average of the absolute values of the differences between the a^ g. 



05/26/2004, EAST Version: 1.4.1 



us 6,690,838 B2 



17 



18 



Similarly, the processor 88 filters the values of the pixels 
(^1^°^° "7 of block A), by 0 (column 0 of block B), and 
by4 (column 1 of block B) for y=0,l^, ... ,7 according to 
the following equations: 

0.14*<,.iv^/a«rf (28) 



0.2Sbf^,,y^^j^^ (29) 

(30) 



0.652),, 



where a*^ ^ corresponds to hpl^^^^^ and thus equals the 
filtered Y value of the pixel a^ 7,b*y q corresponds to 
hp2^^^ and thus equals the filtered Y vahie of the pixel 
b^ o» b'y 1 corresponds to hp3^^„j, atKl thus equals the 
filtered Y value of the pixel b„ 

Likewise, referring to FIGS. 6 and 8B, if the boundary 
124 is blocky, the processor 88 filters the values of the pixels 
ag^ for x-0,1,23, . . . , 7 (row 6 of block A) according to the 
following equation: 

0.05C(i^3^^ (31) 

where a'^^ corresponds to the filtered Y value of the pixel 

VpO, i.e., VpO^^,^ and ^(e^yp^filtend* ^p^)pn-filtervJy *'(0^) 

pre-fiitend^ and ^(^i^^rw-fiiuirvd Tcspectively correspond to the 
pre-filtered Y values of the pixels vp0-vp3, i.e., vpO^. 
yizrrnrd, vpl^^^„^ vp2p^.p^^j, and vp3^^^«^^, of FIG. 
8B. Thus, a'g^ equals the filtered value of the pixel a^^ 

Similarly, the processor 88 filters the values of the pixels 
^7a> ("^^ ^ blodc A), Cq^ (row 0 of block C), and c^^ 
(row 1 of block C) for x=0,l,2, . . . , 7 according to the 
following equations: 

0-25C(i^3^^ (33) 
c'i^-/v3-0.05a(^]p^^/ie,«rf+0.1fl(7^^^jsh5,erf+0.2c^ 



(34) 



where a'^^ corresponds to vpl^^^^^ and thus equals the 
filtered Y value of the pixel bl-j^ Cq^ corresponds to 
^^fiit»r^ ^ tbus equals the filtered Y value of the pixel 
Cq^ and corresponds to vpS^^^ and thus equals the 
filtered Y value of the pixel c^^. 

To implement the above filtering equations, the processor 
88 conventionally stores the pre-filtered values of the pixels 
being filtered. 

From the foregoing it will be appreciated that, although 
specific embodiments of the invention have been described 
herein for purposes of illustration, various modifications 
may be made without deviating from the spirit and scope of 
the invention. 

What is claimed: 

1. An image processing circuit, comprising: 

a processor operable to, 

receive at least a portion of an image, the portion 
including a boundary, first and second contiguous 
pixels disposed on opposite sides of the boundary, 
and a third pixel, the first, second, and third pixels 
having respective first, second, and third pixel val- 
ues; 

generate a boimdary value from the first, second, and 
third values; 
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compare tbe boundary value to a comparison value; and 
reduce a difierence between the first and second values 
if the boundary value has a specified relationship to 
the comparison value. 
5 2. The processing circuit of claim 1 wherein: 

the imagp portion includes a fourth pixel that has a fourth 
pixel value and that is contiguous with and located on 
an opposite side of the boimdary fi-om the third pixel: 
and 

10 the boundary value equals an average of the difference 
between the first and second values and the difference 
between the third and fourth vahies. 

3. The processing circuit of claim 1 wherein: 

tbe imagp portion includes a fourth pixel that has a fourth 
pixel value and that is contiguous with and located on 
an opposite side of the boundary from the third pixel; 
and 

the boundary value equals the lesser of a predetermined 
value or an absolute value of the average of the 
difference between the first and second values and the 
difference between the third and fourth values. 

4. The processing circuit of claim 1 wherein the boundary 
value equals a roughness value that indicates the roughness 
of the boundary. 

25 5. The processing circuit of claim 1 wherein the image 
includes first and second contiguous pixel blocks disposed 
on opposite sides of the boundary, the first and second pixels 
respectively disposed in the first and second blocks, the third 
pixel disposed in the first or second block. 

3Q 6. The processing circuit of claim 1 wherein the first, 
second, and third values of the first, second, and third-pixels 
comprise respective luminance values. 

7. An image processing circuit, comprising: 

a storage circuit operable to store at least a portion of an 

35 image including a boundary, first and second contigu- 
ous pixels disposed on opposite sides of the boundary, 
third and fourth pixels disposed on the same side of the 
boundary as the first pixel, and fifth and sixth pixels 
disposed on the same side of the boundary as the 

40 second pixel, the first, second, third, fourth, fifth, and 
sixth pixels having respective first, second, third, 
fourth, fifrh, and sixth pixel values; and 
a processor coupled to the storage circuit, the processor 
operable to generate a boundary value from the first and 

45 second values, to generate a comparison value as a 
function of the first, second, third, fourth, fifth, and 
sixth pixel values, to compare the boundary value to the 
comparison value, and to reduce a difference between 
the first and second pixels values if the boundary value 

SO is related to tbe comparison value in a specified manner. 

8. The processing circuit of claim 7 wherein the boundary 
value equals tbe difference between the first and second 
values. 

9. The processing circuit of claim 7 wherein the boundary 
55 value equals a roughness value that indicates the roughness 

of the boundary. 

10. The proces^ng circuit of claim 7 wherein the image 
includes first and second contiguous pixel blocks disposed 
on oppo^te sides of the boundary, the first, third, and fourth 

60 pixels disposed in the first block and the second, fifth, and 
sixth-pixels disposed in the second blocks. 

11. The processing circuit of claim 7 wherein the first, 
second, third, fourth, fifth, and sixth pixel values comprise 
respective luminance values. 

65 12. The image processing circuit of claim 7 wherein: 
the image portion further includes a seventh pixel dis- 
posed on the same side of the boundary as the first pixel 
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aod an eighth pixel disposed on the same side of the 
boundary as the second pixel, the seventh and eighth 
pixels respectively having seventh and eighth pixel 
values; 

the first and third pixels are contiguous in a dimension that 
is perpendicular to the boundary; 

the fourth and seventh pixels are contiguous in the dimen- 
sion; 

the second and fifth pixels are cootiguous in the dimen- 
sion; 

the sixth and eighth pixels are continuous in the dimen- 
sion; and 

the processor is operable to generate the comparison value 
as a fiinction of, 

the average of the difference between the first and third 

pixel values and the difference between the fourth 

and seventh pixel values, and 
the average of the difference between the second and 

fifth pixel values and the difference between the sixth 

and eighth pixel values. 

13. The image processing circuit of claim 7 wherein the 
third, fourth, fifth, and sixth pixels are spaced from the 
boundary. 

14. An image processing circuit, comprising: 

a storage circuit operable to store at least a portion of an 
image, the portion including a boundary aod first and 
second contiguous pixel blocks disposed on opposite 
sides of the boundary, each block including pixels 
arranged in alignments and orthogonal alignments and 
having respective values, each pixel in a first aligiunent 
of the first block being contiguous with a respective 
pixel in a first alignment of the second block, the first 
alignments of the first and second blocks each includ- 
ing at least two respective pixels; and 

a processor coupled to the storage circuit, the processor 
operable to generate a boundary value from the values 
of all of the pixels in the first alignments of the first and 
second blocks, to compare the boundary value to a 
comparison value, and to reduce a difference between 
the value of a first pixel in the first alignment of the first 
block and the value of a respective first pixel in the first 
alignment of the second block if the boundary value has 
a specified relationship to the comparison value. 

15. The processing circuit of claim 14 wherein the pixel 
alignments of the first and second blocks each comprise a 
respective coliunn of pixels and the orthogonal pixel align- 
ments of the first and second blocks each comprise a 
respective row of pixels, 

16. The processing circuit of claim 14 wherein the pixel 
alignments of the first and second blocks each comprise a 
respective row of pixels and the orthogonal pixel alignments 
of the first and second blocks each comprise a respective 
column of pixels. 

17. A processing circuit, comprising: 

a storage circuit operable to store at least a portion of an 
image, the portion including a boundary aod first and 
second contiguous pixel blodcs located on opposite 
sides of the boundary, the first and second blocks each 
including respective first and second alignments of at 
least two pixels having respective values, each pixel in 
the first alignment of the first block being contiguous 
with a respective pixel in the first alignment of the 
second block; and 

a processor coupled to the storage circuit, the processor 
operable to generate a boundary value from the values 
of the pixels in the first alignments of the first and 
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second blocks, to generate a comparison value as a 
function of the values of all the pixels in the first and 
second alignments of the first block and in the first and 
second alignments of the second block, to compare the 
boundary value to the comparison value, and to reduce 
a difference between the value of a first pixel in the first 
alignment of the first block and the value of a respective 
first pixel in the first alignment of the second block if 
the boundary value has a specified relationship to the 
comparison value. 

18. The processing circuit of claim 17 wherein the first 
and second ahgnments of the first block and the first and 
second alignments of the second block each comprise a 
respective row of pixels. 

19. The processing circuit of claim 17 wherein the first 
and second alignments of the first block and the first and 
second alignments of the second block each comprise a 
respective column of pixels. 

20. Hie processing circuit of claim 17 wherein the image 
comprises a video image. 

21. The processing circuit of claim 17 wherein the image 
comprises a still image. 

22. A method, comprising: 

calculating a boundary value from a first value of a first 
pixel, a second value of a second pixel, and a third 
value of a third pixel, the first and second pixels being 
contiguous with one another and located on opposite 
sides of a boundary; 
comparing the boundary value to a comparison value; and 
reducing a difference between the first and second values 
if the boimdary value is related to the comparison value 
in a specified manner. 

23. The method of claim 22 wherein calculating the 
boundary value comprises setting the boimdary value equal 
to an absolute value of an average of the difference between 
the first and second values and the difference between the 
third value and a fourth value of a fourth pixel, the third and 
fourth pixels being contiguous and disposed on opposite 
sides of the boundary. 

24. The method of claim 22 wherein calculating the 
boundary value comprises setting the boundary value equal 
to the lesser of a predetermined value or an absolute value 
of an average of the difference between the first and second 
values and the difference between the third value and a 
fourth value of a fourth pixel, the third and fourth pixels 
being contiguous and disposed on opposite sides of the 
boundary. 

25. A method, comprising: 

calculating a boundary value from the values of all the 
pixels in first and second afignments of an image, the 
first and second alignments located on opposite sides of 
a boundary and each including at least two respective 
pixels, each pixel in the first ahgnment being contigu- 
ous with a respective pixel in the second alignment; 
comparing the boundary value to a comparison value; and 
reducing a difference between the value of a first pixel in 
the first alignment and the value of a respective second 
pixel in the second alignment if the boundary value has 
a specified relationship to the comparison value. 

26. TTie method of claim 25 wherein the alignments and 
orthogonal alignments of the first and second blocks com- 
prise respective rows and columns of pixels. 

27. The method of claim 25 wherein the alignments and 
orthogonal alignments of the first and second blocks com- 
prise respective columns and rows of pixels. 

28. The method of claim 25 wherein the values of the first 
and second pixels comprise respective liuninance values. 
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29. An image prooessiDg circuit, comprising: 
a processor operable to, 

receive at least a portion of an image, the portion 
including a predetermined boundary and first and 
second contiguous pixels disposed on opposite sides 
of the boundary and third and fourth contiguous 
pixels disposed on opposite sides of the boundary, 
the first, second, third, and fourth pixels having 
respective first, second, third, and fourth pixel val- 
ues; 

determine whether the boundary is blocky as a function 
of the first, second, third, and fourth pixel values; and 

reduce a difference between the first and second pixel 
values and a difference between the third and fourth 
pixel values if and only if the boundary is blocky. 

30. A method, comprising: 

determining whether a predetermined boundary between 
first and second contiguous pixels and between third 
and fourth contiguous pixels is blocky using the values 
of the first, second, third, and fourth pixels; and 

reducing a difference between the first and second values 
and reducing a difference between the third and fourth 
values if and only if the boundary is blocky. 

31. An image processing circuit, comprising: 
a processor operable to, 

receive at least a portion of a decoded image, the 
portion including a boundary between first and sec- 
ond contiguous pixels; and 

calculate a roughness of the boundary as a function of 
quantization during encoding of an image, the 
encoding producing an encoded image that is later 
decoded into the decoded image. 

32. A method, comprising: 

identifying a boundary between contiguous pixels of a 

decoded image; and 
calculating a roughness of the boundary as a function of 

quantization that is performed during encoding of an 

image, the encoding producing an encoded image that 

is later decoded into the decoded image. 

33. An image processing circuit, comprising: 
a processor operable to, 

receive at least a portion of an image that has been 
decoded iising a block-based decoding algorithm, the 
portion including a boundary and first and second 4S 
contiguous decoded pixel blocks disposed on oppo- 
site sides of the boimdary, the first pixel block having 
a first alignment of pixels and the second pixel block 
having a second alignment of pixels that is contigu- 
ous with the first alignment, the pixels within the first 
and second alignments having respective pixel val- 
ues; 

determine whether the boundary is blocky as a function 
of an average difference between the value of each 
pixel in the first alignment and the value of a 
respective contiguous pixel in the second alignment; 
and 

reduce a difference between the first and second pixel 
values if and only if the boundary is blocky. 

34. A method, comprising: 
for a boundary between first and second contiguous pixel 

blocks that have been decoded according to a blodc- 
based decompression standard, the first pixel block 
having a first alignment of pixels and the second pixel 
block having a second alignment of pixels that is 6S 
contiguous with the first alignment, the pixels within 
the first and second alignments having respective pixel 
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values, determining whether the boundary is blocky as 
a function of an average difference between the value 
of each pixel in the first alignment and the value of a 
respective contiguous pixel in the second alignment; 
and 

reducing a difference between the first and second values 
if and only if the boundary is blocky. 

35. A method, comprising: 

determining whether a boundary between contiguous first 
and second pixel blocks is blocky, the pixel blocks 
having been decoded according to a block-based 
decompression standard, the first pixel block having a 
first pixel and the second pixel block having a second 
pixel that is contiguous with the first pixel, the first and 
second pixels having respective first and second values; 

reducing a difference between the first and second values 
if and only if the boundary is blocky; 

wherein the first pixel block includes a first orthogonal 
alignment of pixels that includes the first pixel; 

wherein the second pixel block includes a second 
orthogonal alignment of pixels that includes the second 
pixel and that is contiguous with the first orthogonal 
alignment; and 

wherein the processor is operable to determine whether 
the boundary is blocky by, 

calculating a boundary roughness equal to the average 
difference between the values of contiguous pairs of 
pixels in the first and second afignments, 

calculating a first-block average pixel difference equal 
to the siun of the differences firom the first block 
divided by the number of differences from the first 
block, 

calculating a second-block average pixel difference 
equal to the sum of the differences from the second 
block divided by the number of differences fi'om the 
second block, 

calculating a first-block average pixel vahie equal to the 
simi of the values of the pixels in the first block 
divided by the number of pixels in the first block, 

calculating a second-block average pixel value equal to 
the sum of the values of the pixels in the second 
block divided by the number of pixels in the second 
block, 

calculating a comparison value equal to the sum of the 
product of a first factor and the sum of the first- and 
second-block average pixel differences and the prod- 
uct of a second factor and the sum of the first- and 
second-block average pixel values, and 

determining that the boundary is blocky if the boundary 
roughness has a predetermined relationship to the 
comparison value. 

36. A method, comprising: 

determining whether a boundary between contiguous first 
and second pixel blocks is blocky, the pixel blocks 
having been decoded according to a block-based 
decompression standard, the first pixel block having a 
first pixel and the second pixel block having a second 
pixel that is contiguous with the first pixel, the first and 
second pixels having respective first and second values; 

reducing a difference between the first and second values 
if and only if the boundary is blocky; 

wherein the first pixel btock includes a first orthogonal 
alignment of pixels that includes the first pixel and a 
second orthogonal alignment of pixels that is contigu- 
ous with the first alignment; 

wherein the second pixel block includes a third orthogo- 
nal alignment of pixels that includes the second pixel 
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aod that is contiguous with the first orthogonal 
alignment, and a fourth orthogonal alignment of pixels 
that is contiguous with the third alignment; and 
wherein the processor is operable to determine whether 
the boimdary is blocky by, 5 
calculating a boundary roughness equal to tbe mini- 
mum of a first predetermined value and the average 
difference between the values of contiguous pairs of 
pixels in the first and third alignments, 
calculating a fiist-block average pixel difference equal 
to tbe sum of the differences between the values of 
the corresponding pixels in the first and second 
alignments divided by the number of differences 
from the first and second alignments. 
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calculating a second-block average pixel difference 
equal to tbe sum of the differences between the 
values of the corresponding pixels in the third and 
fourth alignments divided by the number of differ- 
ences firom the third and fourth alignments, 

calculating a comparison value equal to the siun of a 
second predetermined value and the product of a first 
factor and the maximum of tbe first-block pixel 
average and the second-block pixel average, and 

determining that the boundary is blocky if the boundary 
roughness has a predetermined relationship to the 
comparison value. 
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